Method for minimal-logic non-linear filter implementation

ABSTRACT

A method for implementing a filter on a signal is disclosed wherein interval membership information is computed and stored in such a manner so as to use a minimal amount of memory thereby allowing filter computation by a small number of deterministic sequence of table lookups and bit-wise logical operations. In general, the present invention involves using a non-linear filter represented as a plurality of intervals and the discrete values a sample may take. Each sample corresponds to a component in a vector. Each interval is comprised of a lower and upper vector. Each sample value is compared to the lower and upper values of the sample&#39;s associated coordinate. A table for that sample is then constructed with binary entries where a ‘1’ denotes that a sample value is within the lower and upper values of the coordinate and ‘0’ denoting otherwise. A table is built for each sample.

This application is a continuation-in-part of U.S. application Ser. No.09/561,609, filed Apr. 27, 2000, now U.S. Pat. No. 6,816,621 dated Nov.9, 2004 the disclosure of which is totally incorporated by reference.

FIELD OF THE INVENTION

The present invention is generally related to the field of signalprocessing and, more particularly, to the field of programmable hardwareand/or software filtering systems and methods of image data in compact,efficient computing architectures.

BACKGROUND OF THE INVENTION

In the arts it is often necessary to manipulate a received signal orimage, for both practical and operational reasons. For instance, thesignal may need to be compressed into a plurality of smaller subsetsprior to some secondary operation being performed on the data such astransmission over a cable or phone wire. On the receiving end,compressed transmitted signals may eventually need to be restored orreconstructed to return the signal upon receipt to its original form.Other operations can take the form of feature detection or patternrecognition, among others. In order to manipulate the received signal orimage data in these instances, mathematical operations are oftenperformed. To achieve this, the discrete signal or image is oftenconverted into or represented by a mathematical sequence or array ofdiscrete values. For example, binary signals or images often take theform of discrete values of [0,1] and 8-bit gray-scale images often takethe form of values in the range of [0.255]. The field of discrete signaland image processing, both theoretical and applied, is relativelydominated by mathematics in the form of specific methods andapplications.

Many methods of signal or image processing use one or more applicationsinvolving filters. A filter operates by capturing a windowed observationof an image or signal. A windowed observation is a collection or vectorof samples from the captured image or signal. Based on this vector, adecision, based on some formula or algorithm, is made about the image orsignal at a particular position within the image or signal. In general,a filter operates on a set of signal or image samples which aremathematically written as a vector having the form of (X₁, . . . ,X_(N)).

Many techniques can be found in the arts, of which the following arerepresentative and which are hereby incorporated by reference being madethereto. For example, a filter could be employed to assign one or morespecific code-words to a given sample as shown in FIG. 1, (often calledvector quantization), as taught by Vector Quantization and SignalCompression, A. Gesho and R. M. Gray, 1991. A filter can also assign oneor more tags to a center pixel in order to indicate that the vectorrepresented pixels are part of a specific feature of the captured image,such as a centerline, letter, or symbol. It can also assign a new samplevalue to the center pixel in order to accomplish the overall goal of arestoration or enhancement of a degraded image, as taught by Enhancementand Restoration of Digital Documents, R. P. Loce and E. R. Dougherty,SPIE Press, 1997, and Enhancement of Digital Documents, R. P. Loce andE. R. Dougherty, Electronic Imaging Technology, SPIE Press 1999, andTwo-Dimensional Signal and Image ProcessingJ. S. Lim, Prentice Hall,1990. A filter can also be used to predict a sample or pixel valueoutside the capture window with the aim of reducing the amount of dataneeded to store the image. This technique is often referred to aspredictive encoding as taught by Digital Image Compression Techniques,M. Rabbani and P. W. Jones, SPIE Press, 1991. A filter can be used toassign an array of fewer, more restrictive values to this observation(often referred to as either “quantization,” “thresholding,” or“dithering” as taught by Digital Halftoning for Printing and Display ofElectronic Images, R. P. Loce, P. G. Roetling, and Y. W. Lin, ElectronicImaging Technology, SPIE Press, 1999. Many others are found in the knownliterature. Other applications of filters used in signal or imageprocessing include, resolution conversion, object detection,speckle-removal, and edge enhancement.

Nonlinear image or signal processing is a general representation ofsignal or image filtering based on a logical decomposition of a filterinto a set of relatively simple operators. Any windowed shift-invariantfilter can be represented as a combination of simple operations calledhit-or-miss transforms as taught by Nonlinear Filters for ImageProcessing, E. R. Dougherty and J. Astola (eds.), SPIE/IEEE Press, 1999.The term nonlinear refers to the entire class of windowed filters thatincludes linear filters, i.e., those filters computed by convolution.

Since filters can be highly complex and repeatedly be executed millionsof times in rapid succession in order to perform a single operation on agiven image or signal, what is needed in the arts are compact, efficientcomputing architectures directed towards this end. Efficientarchitectures depend on filter representations. If a filter can bebroken down into its base operations, it might be possible to executethe relatively simple operations in parallel in order to achievehigh-speed execution and overall efficiency.

Computational Gray-Scale Morphology on Lattices (A Comparator-BasedImage Algebra) Part I: Architecture, E. R. Dougherty and D. Sinha, RealTime Imaging, Vol. 1, No. 1, 1995, teaches a computer architecture usingrepresentations that perform tests in parallel. The representationalmethods as taught therein express any filtering operation as acollection of logical tests or operations wherein each logical test isassociated with a value. If a windowed observation satisfies a test, thefilter output is then associated with that value. Although the abovearchitecture is not practical for filters requiring many tests, it doesteach general representational method.

It is the aim of the present invention to overcome the conventionalprior art print limitations, as described above, by programming acomputational architecture to perform logical comparisons in a morecompact and efficient way.

SUMMARY OF THE INVENTION

A method for implementing a filter on a signal is disclosed whereininterval membership information is computed and stored in such a mannerso as to use a minimal amount of memory thereby allowing filtercomputation by a small number of deterministic sequence of table lookupsand bit-wise logical operations. In general, the present inventioninvolves using a non-linear filter represented as a plurality ofintervals and the discrete values a sample may take. Each samplecorresponds to a component in a vector. Each interval is comprised of alower and upper vector. Each sample value is compared to the lower andupper values of the sample's associated coordinate. A table for thatsample is then constructed with binary entries where a ‘1’ denotes thata sample value is within the lower and upper values of the coordinateand ‘0’ denoting otherwise. A table is built for each sample.

More specifically, the method of the present invention involves thesteps of first obtaining a windowed observation from an image or signaland storing that observation in a vector X such that X=(X₁, . . .,X_(N)). Then, for each component X_(i) stored in vector (X₁, . . .,X_(N)), obtaining an index into a table to produce a one-dimensionalinterval number L_(i)(X_(i)). For each produced one-dimensional intervalvalue L_(i)(X_(i)), utilizing that number as an index into a table ofbit-vectors to obtain the set of corresponding bit-vectors. Performing alogical AND operation for each set of N bit-vectors. Retrieving theposition k of a unique non-zero bit and using position k as an indexinto a second table T of stored filter values. Retrieving thecorresponding stored filter value such that T[k] is the filter value atobservation X.

Additional benefits and advantages of the present invention will be setforth in the description which follows, and in part will be obvious fromthe description, or may be learned by the practice of this invention.The advantages of this invention, as described herein, may be realizedand obtained by means particularly pointed out and distinctly claimed inthe appended claims, taken in conjunction with the accompanying drawingsand detailed specification.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the manner in which the above-recited and other advantagesand objects of the present invention are obtained, a more particulardescription of this invention, briefly described above, will be renderedby reference to a specific embodiment thereof which is illustrated inthe appended drawings. Understanding that these drawings depict only atypical embodiment of the present invention and are not therefore to beconsidered in any way limiting of its scope, this invention will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings, in which:

FIG. 1 is an illustration of a segment of a window capture wherein eachpixel or sample is represented by vector notation based on a windowedobservation (x₁, x₂, x₃, x₄, x₅, x₆, x₇, x₈, x₉) to modify or tag acenter pixel;

FIG. 2 illustrates a filter represented a sequence of logical testsfollowed by a table lookup;

FIG. 3 illustrates a two-pixel filter represented geometrically as apartition of the set of all possible pixel-value pairs;

FIG. 4 illustrates the five intervals of the filter represented in FIG.3;

FIG. 5 illustrates the quantization step for the filter of FIG. 3;

FIG. 6 illustrates the minimal bit-vector storage for the filter of FIG.3; and

FIG. 7 illustrates the data flow and decision logic for a nonlinearfilter in the accordance with the techniques of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

What is presented is method for implementing a minimal architecture forcomputing a non-linear filter that performs a set of logical comparisonsin a more compact and efficient manner so as to reduce computationaltime and system memory requirements.

At the onset, it should be understood that one skilled in this artshould have a solid background in higher mathematics specifically in thefields of nonlinear signal or image processing, and be readily familiarwith the concepts of binary operations, basic filtering concepts andoperations, and preferably computer architecture.

A filter, whether operating on binary, gray-scale or other multilevelsignals, can be represented mathematically by a set of intervalmemberships as taught in: Computational Gray-Scale Morphology onLattices (A Comparator-Based Image Algebra) Part I: Architecture, E. R.Dougherty and D. Sinha, Real Time Imaging, Vol. 1, No. 1, 1995,incorporated herein by reference. To brieflv explain, a filter can berepresented mathematically as a function which maps a vector X to anoutput value a. Let us call S[a] the set of all vectors X which map tooutput value a. Each output value a of the filter thus has a set S[a] ofall mapping vectors X. As a result, every possible signal sample from Xbelongs to S[a] for some a. If X belongs to S[b], for example, then thefilter value at X is b. Now, each set S[a] can be further partitionedinto a set of intervals [A,B] where interval A is less than or equal tointerval B. Evaluating the filter is equivalent to determining to whichinterval among a plurality of intervals observation X belongs, and thengiven that interval, finding which set S[a] the interval belongs. Thusthe problem in the arts to which the present invention is directed isthe step of finding which interval a signal observation X belongs amonga plurality of intervals. This can be a rather difficult problembecause, in part, each interval either must be tested in turn therebyrequiring much computational time or the entire set of intervals must betested in parallel thereby requiring much computer memory. The step offinding the S[a] the set of all vectors X which map to output value agiven an interval thereafter comprises a table lookup.

A method for implementing a filter on a signal is disclosed whereininterval membership information is computed and stored in such a mannerso as to use a minimal amount of memory thereby allowing filtercomputation by a small number of deterministic sequence of table lookupsand bit-wise logical operations. In general, the present inventioninvolves using a non-linear filter represented as a plurality ofintervals and the discrete values a sample may take. Each samplecorresponds to a component in a vector. Each interval is comprised of alower and upper vector. Each sample value is compared to the lower andupper values of the interval's associated coordinate. A table for thatsample is then constructed with binary entries where a ‘1’ denotes thata sample value is within the lower and upper values of the coordinateand ‘0’ denoting otherwise. A table is built for each sample.

The method of the present invention involves the steps of firstobtaining a windowed observation from an image or signal and storingthat observation in a vector X such that X=(X₁, . . . , X_(N)). Then,for each component X_(i) stored in vector (X₁, . . . , X_(N)), obtainingan index into a table to produce a one-dimensional interval numberL_(i)(X_(i)). For each produced one-dimensional interval valueL_(i)(X_(i)), utilizing that number as an index into a table ofbit-vectors to obtain the set of corresponding bit-vectors. Performing alogical AND operation for each set of N bit-vectors. Retrieving theposition k of a unique non-zero bit and using position k as an indexinto a second table T of stored filter values. Retrieving thecorresponding stored filter value such that T[k] is the filter value atobservation X.

Attention is directed to FIG. 1 which illustrates a windowed observationof a captured gray-scale image wherein nine samples or pixelsrepresented by vector notation. From these nine samples of capturedpixels, a filter is employed to make one or more decisions (orobservations) with respect to the entire captured gray scale image. Insignal or image processing, a filter operates on a sampled signal orimage. In gray-scale image processing, each sample typically is sampledinto one of 256 values represented as an eight-bit value. A filteroperates on a plurality of simultaneous samples. Typically, non-linearfilters are represented as a sequence of logical tests as to whether ornot a windowed observation of a signal or captured image falls betweentwo distinct values.

Given an observation with Nsamples, as represented by X=(X₁, . . . ,X_(N)), and the desire to test whether X is between a lower and an uppervalue given by A=(A₁, . . . , A_(N))≦X≦B=(B₁, . . . , B_(N)), whereincomponent-wise: A_(i)≦X_(i)≦B_(i) for each i=1, . . . , N, a filter canbe employed to represent a list of the upper and lower pairs [A, B] andeach interval has its associated output value. Once it is determined towhich interval an observation belongs, the output value of the filtercan be determined from this interval. With reference still being made toFIG. 1, a 3×3 sample is taken resulting in a vector of nine samplesrepresented by X=(X₁, . . . ,X₉). A filter maps this vector to one of aplurality of values. In general, the observation window can contain anarbitrary but finite number of observations although it need not besquare nor contiguous. The general observation is denoted by X=(X₁, . .. ,X_(N)) where N is unspecified. There are many operations contemplatedin the art, including smoothing, restoration, quantization, among manyothers, for instance, a digital filter with a finite number of possibleoutput values can be represented as a process whereby the sample X iscompared to a plurality of intervals, or pairs of lower and uppervectors given by [A₁,B₁], . . . , [A_(M),B_(M)]. Each interval[A_(j),B_(j)] has an associated filter output value. The filter comparesan observation X with each interval in a list until the first intervalis found such that A_(j)<=X<=B_(j). This may take as few as onecomparison or as many as M comparisons. Once an interval is found, thevalue associated with it in the table is then returned as the value ofthe filter at X. This sequential evaluation process is illustrated inFIG. 2.

FIG. 3 illustrates a simple example filter using only two observations Xand Y. Any observation (x, y) falls into one and only one of fiveintervals as indicated. For example, (200, 100) falls into interval L₂.This interval has a filter value associated with it. Determining thefilter value for each observation requires a non-deterministic number ofsteps and requires a significant amount of computation. The presentinvention produces a filter value in a fixed number of computationalsteps and also uses a minimal amount of computer memory.

FIG. 4 shows the coordinates of the five intervals that comprise thefilter in FIG. 3 from which it can be observed that each two-dimensionalinterval can be projected onto the X and Y axes to produce threeone-dimensional intervals on each respective axis. For example, intervalL₄ produces interval [64,191] on the X axis. For each axis, label theseone-dimensional intervals from 0 to the maximum number minus 1. For agiven value of sample x, x fits into one of these intervals. Similarly,for a given value of y, y fits into one of the intervals of the Y axis.

FIG. 5 shows a table for each axis and the one-dimensional intervallabels for each sample value. For each axis, each one-dimensionalinterval corresponds to a number of two-dimensional intervals. Forexample, one-dimensional interval [64,191] on the x-axis corresponds totwo-dimensional intervals L₁, L₃, and L₄ because points in thesetwo-dimensional intervals project to points in the interval [64,191] onthe x-axis.

FIG. 6 shows a table for each axis of the filter in FIG. 3. The columnrepresents a label for the one-dimensional interval and the row entryrepresents labels of the two-dimensional intervals. A table has entry‘1’ in position (i,j) if one-dimensional interval i corresponds totwo-dimensional interval j. For example, row three of the table for thex-axis is 0 1 0 1 1 which shows that one-dimensional interval 1corresponds to two dimensional intervals 1, 3, 4. It is a consequence ofthis design that the rows or bit-vectors in each table are uniqueregardless of the number of axes, the possible samples per pixel (e.g.,8, 12, etc.), or N-dimensional intervals. Thus the tables requireminimal storage.

Given an observation, for example (200, 100), each sample is indexedinto the one-dimensional interval table corresponding to its axis. Inthis example, sample x=200 is indexed into the x-axis table in FIG. 5 toproduce label 2 and sample y=100 is indexed into the y-axis table inFIG. 5 to produce label 1. X-axis label 2 is indexed into the x-axisbit-vector table to produce 0 0 1 1 0 and y-axis label 1 is indexed intothe y-axis bit-vector table to produce 1 0 1 0 1. These two bit-vectorsare logically AND'd together to produce 0 0 1 0 0. The bit position ofthe unique ‘1’ is position 2 (counting from 0 from the left). Thisposition is used to index a last table to produce the filter value for(200,100).

FIG. 7 illustrates the present invention in the general case where thereare N-samples per observation.

In summary, a method for implementing a filter on a signal is disclosedwherein interval membership information is computed and stored in such amanner so as to use a minimal amount of memory thereby allowing filtercomputation by a small number of deterministic sequence of table lookupsand bit-wise logical operations. In general, the present inventioninvolves using a non-linear filter represented as a plurality ofintervals and the discrete values a sample may take. Each samplecorresponds to a component in a vector. Each interval is comprised of alower and upper vector. Each sample value is compared to the lower andupper values of the sample's associated coordinate. A table for thatsample is then constructed with binary entries where a ‘1’ denotes thata sample value is within the lower and upper values of the coordinateand ‘0’ denoting otherwise. A table is built for each sample.

Windowed shift-invariant filters represent a plethora of operationsincluding but not limited to processing sampled digital electricalsignals such as used in sensors, sound reproduction, servo-mechanicalcontrol, audio data compression, ultrasound, and so on. In twodimensions, windowed shift-invariant filters are used to process imagedata in medical imaging, remote sensing, machine vision and inspection,document processing, synthetic aperture radar, military targetrecognition, and so on. Examples are object detection, imageenhancement, data compression, thresholding, noise-removal, imagereconstruction, resolution conversion, segmentation, watermark insertionand detection, character recognition, and various data manipulations toprepare images for printing or display. Samples may also be obtainedfrom higher-dimensional data such as voxels in a digital threedimensional representation of materials. Furthermore, the presentinvention will find its implementations in either hardware or computersoftware depending on the user's specific requirements. Windowedshift-invariant filters represent a plethora of operations including butnot limited to processing sampled digital electrical signals such asused in sensors, sound reproduction, servo-mechanical control, audiodata compression, ultrasound, and so on. In two dimensions, windowedshift-invariant filters are used to process image data in medicalimaging, remote sensing, machine vision and inspection, documentprocessing, synthetic aperture radar, military target recognition, andso on. Examples are object detection, image enhancement, datacompression, thresholding, noise-removal, image reconstruction,resolution conversion, segmentation, watermark insertion and detection,character recognition, and various data manipulations to prepare imagesfor printing or display. Samples may also be obtained fromhigher-dimensional data such as voxels in a digital three dimensionalrepresentation of materials. Further, images may have multiple channels,for example, color images may have three or four channels. Remotelysensed images may be hyper-spectral and have data in tens of spectralbands, for example, a hyper-spectral device may have a plurality ofsensors, each sensitive to a specific interval of wavelengths such aslong infrared, near infrared, visible, ultraviolet, etc. wherein eachsensor may generate signals from electromagnetic radiation in itsrespective band of sensitivity. Windowed shift-invariant filters can beapplied simultaneously on all image bands using the method of thepresent invention. For example, a filter may have some samples from along-wave infrared band, some samples from a near infrared band, somefrom the visible portion of the light spectrum, and some samples fromultraviolet. When taken together, a windowed observation consists ofsamples separated in space or time as well as wavelength. In addition,the method of the present invention readily finds its implementations onsampled signal or image data whether the data are from spatiallyarranged sensors or from sensors tuned to different wavelengths or acombination thereof.

With the above-described detailed description of the preferredembodiment of the present invention and the above-described intendedusages and envisioned variations thereto, one skilled in the art ofcomputer architecture and programming and filter design andimplementation will readily find their specific implementation inaccordance herewith.

This invention may be embodied in other specific forms without departingfrom its spirit or essential characteristics. The above-describedembodiments are to be considered in all respects only as illustrativeand not restrictive in scope. The scope of the invention is, therefore,indicated by the appended claims rather than by the above-detaileddescription. Therefore, all changes which come within the meaning andrange of equivalency of the claims are to be considered embraced withintheir scope.

1. A method for implementing a filter on a signal wherein intervalmembership information is computed and stored in such a manner thatfilter computation by a small number of deterministic sequence of tablelookups and bit-wise logical operations can be performed quickly andefficiently, comprising the steps of: a) obtaining a windowedobservation from an image or signal and storing that observation in avector X such that X=(X₁, . . . ,X_(N)); b) for each component X_(i)stored in vector (X₁, . . . , X_(N)), obtaining an index into a table toproduce a one-dimensional interval number L_(i)(X_(i)); c) for eachproduced one-dimensional interval value L_(i)(X_(i)), utilizingL_(i)(X_(i)) as an index into a table of bit-vectors to obtain the setof corresponding bit-vectors; d) performing a logical AND operation foreach set of N corresponding bit-vectors; e) retrieving position k of aunique non-zero bit and using position k as an index into a second tableT of stored filter values; and f) de-referencing the correspondingstored filter value such that T is the filter value at observation X.